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1. INTRODUCTION 

Communication is very important in all areas of life. Its significance is considered by many people 
to be as important as breathing. It aids in the process of sharing information and also improves the 
relationship with other people. Unfortunately, not all are blessed with the ability to talk and hear. There are 
some who cannot talk nor hear what others are saying, in fact,the World Federation of the Deaf published a 
report that 72 million deaf-mute people [1] make up the world’s population. 

In order to communicate with the society, researchers studied how to recognize hand gestures and 
there are two well-known approaches; image processing [2] and the data glove. The data glove approach has 
less computational time and faster response in real time [3] applications which makes it more advantageous 
than image processing technique. As a result of these advantages, there has been quite a reasonable amount 
of research work in building the data glove using different input components like flex sensor [1]-[9], 
accelerometer [4], surface electromyography [4] andMicro-Electro-Mechanical Systems (MEMS) sensor 
[10]. While all of these data gloves can recognize the American Sign Language (ASL) alphabet, few have 
designed it to recognize ASL sign words. No one has also looked into using the data glove in Deaf 
Education. 

Hearing-impaired children who didn’t learnto speakalso go to schools that teach them howto 
communicate with other people. In educating deaf and mute children, there are three great approaches that 
can be successfully used: the bilingual-bicultural (BiBi) approach which makes use of ASL; the auditory- 
verbal approach that teaches the English Language through residual hearing and speech instead of sign 
language; and total communication that combines auditory and visual communication for instruction. These 
techniques has under gone waves of popularity, but this doesn't mean though that one is better than the other. 
Hearing-impaired children are different and each needs a different type of technique to help them with their 
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development of language and communication. However, ASL, which wasthe first language ever taught in the 
BiBi approach, is said to be the native language for them and is historically used in Deaf Education. 

Keeping in mind the use of BiBi approach in teaching hearing-impaired children, technology can be 
integrated across the curriculum to assist professionals and teachers in Deaf Educationand make learning 
more interesting. Technology has been digitizing classrooms through digital learning tools that have 
increased student’s engagement and motivation towards learning. 

This paper explored on the development of a data glove as a digital learning tool for teaching 
hearing-impaired children. The data glove was named Electronic Glove or E-Glove. It was developed using a 
general-purpose microcontroller board for processing input coming from the combination of flex sensors and 
accelerometer. The use of the accelerometer only for recognizing ASL sign words is what separates the E- 
Glove from other existing data gloves. 


2. RESEARCH METHOD 

The primary goal in the development of E-Glove is showing the recognized ASL letters and words 
through a software program installed in a computer. It shall facilitate in teaching hearing-impaired children 
through teachers showing the proper hand gestures and then letting the students perform it using the glove. 

Figure | represents the block diagram of E-Glove. Five flex sensor and one accelerometer were 
connected to the microcontroller that contained the functionality of the device. UHF modules were used to 
wirelessly connect the E-Glove to the computer. Any letter or word recognized using the E-Glovewill be 
shown and read aloud on the computer. 


Flex Sensors 
Accelerometer 


UHF Module 


Microcontroller 


Vv 
UHF Module 


Figure 1. The Block Diagram of E-Glove 


2.1. General description 

Figure 2 shows the connection of the different components used in the development of E- 
Glove.Once the microcontroller detects the sign language, it will send the letter or word that corresponds to 
the hand gesture wirelessly into the computer using UHF (transmitter). The UHF (receiver) connected to the 
UART acts as a bridge between the microcontroller and the computer. The computer will then display the 
letters or words on the screen and also convert it into speech. 


a. Arduino Mega 
b. Flex sensor 
c. Accelerometer 
d. UHF Module 
e. 


Software Program 


2.2. Arduino mega microcontroller 

Arduino microcontroller is a long-established general purpose microcontroller [11] used for 
developing do-it-yourself electronic projects and numerous embedded systems. Arduino microcontrollers are 
physically programmable circuit boards that can be programmed using an Arduino IDE (Integrated 
Development Environment) [12] which is available in all portals [13] that deal with a product. The IDE is 
also used tocompile and upload the written source code to the Arduino board. Arduino is both an open source 
[14] hardware and software. 

There are many types of Arduino microcontroller board. We used the Arduino Mega for 
E-Glove because it has lots of digital input/output pins and analog input pins. This board is very convenient 
for projects that require a lot of digital input and output. The Arduino Mega has the following specifications: 


a. Microcontroller - ATmega1280 
b. Operating Voltage - 5V 
c. Recommended Input Voltage - 7-12V 
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d. Digital I/O Pins - 54 

e. Analog Input Pins - 16 

f. DC Current per I/O Pin - 40 mA 
g. Clock Speed - 16 MHz 


The Arduino Megamicrocontroller wasthe most important component of the E-Glove because it 
contained the whole functionality of the device. Its absence won’t make the device work. It holds the code for 
the sensors as well as the database of the device. It also converts the analog signals coming from the sensor 
into digital in order for the computer to read the data. 

The flex sensors were connected to the analog pins AO-A4 while the accelerometer used A5 of the 
Arduino microcontroller. The first transceiver which served as a transmitter was connected to the Rx and Tx 
communication port of the Arduino microcontroller. 


Figure 2. The circuit diagram of the device 


2.3. Flex sensors 

Flex sensors act as variable resistors mainly used to detect bending or flexing. The resistance of this 
kind of sensors changes when bent. They are made up of carbon on a strip of plastic wherein resistance gets 
higher and higher while bending in one direction. Flex sensors differ in length but most havea resistance 
ranging from about 10 kilo-ohms to 35 kilo-ohms. 

In the E-Glove, the degree of bend of the flex sensor was compared to the mapped valuesin the code 
to recognize aparticular sign language. Five pieces of flex sensors were placed on each finger. Their values 
are being recorded and compared with the values that represent a particular sign language. If the value 
matches, it will be shown in the software program. 


2.4. Accelerometer 

The accelerometer is an inertial sensorthatis dynamically capable of sensing in a vast range. 
Accelerometers are used to measure acceleration forces in one, two, or three orthogonal axes depending on 
its application to a certain device. 
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For the E-Glove to detect hand movements when doing a certain ASL sign word, the accelerometer 
was used. The value read from the accelerometer was compared to the value saved in the software program to 
determine if a certain word corresponds to the hand movement. 


2.5. UHF module 

UHF module was used in the project as a bridge of the E-Glove and the computer for them to 
communicate wirelessly. The UHF module connected to the E-Glovesends the data to the computer. The 
purpose of making the device wireless is to make the user comfortable and not to stay on just one area of the 
room. 

Specifically, UHF-EX was used in the E-Glove. It is a wireless UART transceiver with 100 
milliwatts (mW) radio frequency output. Two UHF-EXwere used in E-Glovewhich served different 
purposes. The first one was used as a transmitter connected to the Arduino microcontroller while the second 
one was used as a receiver connected to the computer. The radio frequency power of UHF-EX allowed a 
useful control range of more than 500 meters while unobstructed line of sight range can reach 700 meters. 


a. Power Input - 5V/3.3V jumper selectable 

b. Frequency Range - 431.1 MHz — 437.3 MHz 

c. Channel Separation - 400kHz 

d. Modulation Type - FSK ( Frequency Shift - Keying ) 
e. UART Baud rate - 9600 bit/s 

f. Current Consumption TX - 36mA @ 5V 

g. Current Consumption RX - 23mA @ 5V 

h. Transmit to Receive Latency - 20 - 30 milliseconds (ms) 


2.6. Software program 

The data coming from the two sensors attached to the E-Glove were wirelessly sent to the computer 
and a software program was developed to receive this data and show the corresponding sign language 
equivalent. Moreover, the equivalent letter or word was also read aloud using a text-to-speech function. 

The software program was developed using Microsoft Visual C# Express. Microsoft Visual C# 
Express is a high-level programming language that is intended for building a variety of application using the 
.NET Framework. This programming language issimple, powerful, type-safe and objected-oriented. The 
continuous development of this language enables rapid application development while retaining the 
simplicity and elegance of a C based programming language. 


2.7. American sign language (ASL) alphabet and sign words 

ASL is a natural language that serves as the first language taught in the BiBi approach of deaf 
education before the English language. One of the first things that a normal person should learn is spelling. In 
deaf education, the ASL alphabet is being taught to hearing-impaired children for them to spell words and 
most especially names. 


Figure 3. The ASL alphabet from North Star Teacher Resources 
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Figure 3 represents the ASL alphabet from North Star Teacher Resources. ASL sign words were 
also taught to hearing-impaired children especially those important words that they need in everyday 
conversation like Know, You, Good, Fine, Understand, Thank you, Again, Me, Hi and Bad. 


3. RESULTS AND ANALYSIS 
3.1. The E-Glove 

The E-Glove shown in Figure 4 was sewn with flex sensors in each finger. The accelerometer was 
positioned on top of the hand. Both flex sensors and accelerometer were connected in the Arduino 
microcontroller which was placed inside a box that is wrapped around the forearm. 


Figure 4. The prototype glove 


The E-Glove sends the data wirelessly to the computer. This feature is not exhibited on most of the 
previous data gloves [2], [4], [6], [8]-[10] but proven beneficial to other designs [1], [3], [5], [7]. The 
wireless feature of E-Glove should encourage a healthy classroom interaction in which the teacher can let the 
students use the E-Glove in their seat and perform a particular hand gesture that may serve as a form of 
recitation in Deaf Education. 


3.2. Sensor values 

Output data were directly obtained from data glove and each sensor produced different resistance 
value depending on the hand gesture. Signals produced by the sensors were translated by the Arduino 
microcontroller into digital form and through serial communication, the data were transferred to the Arduino 
software. 

As seen in Figure 5, the sensors located at the fingers haddifferent readings depending on the sign 
language. Using the Arduino software, the sensor values were mapped to get the specific letter or word. The 
letter or the word which corresponds to the values was sent to the software program. These letters or words 
were then shown in the graphical user interface and at the same time, it was read aloud by the system. 

Figure 6 shows the reading of accelerometer in the Arduino software. On each columns were the 
readings of x, y, and z-axis respectively. The accelerometer was also mapped according to the movement of 
the hand. 
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Figure 5. The Flex Sensor values Figure 6. The Accelerometer values 


3.3. Software program 

The software program developed in Visual C# is shown in Figure 7. It was designed to be user- 
friendly and attractive for children. There were two, large and small, textboxes. The large text box is where 
the accumulated words or letters show up while the smaller text box shows the last letter or word recognized 
by the software program. Buttons were labeled concisely and clearly. The “Start” button enables the software 
program to show the corresponding letter or word on the text boxes based on the gesture exhibited by the 
user. The “Back Space” button deletes the last character in the large text box while the “Clear” button 
removes all the characters in the large text box. The “Stop” button stops the program to show the letter or 
word recognized by the E-Glove. 
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Figure 7. The Graphical User Interface (GUI) of the software program 


The design of the GUI enabled children in Deaf Education to spell out words shown in the large text 
box, something that the previous data gloves lack since they can only show the last recognized letter [1], [4]. 
Moreover, the GUI design is forgiving because it lets the user to correct misspelled words using the 
“Backspace” button.The accumulation of letter in the large text box will enable the teacher to evaluate and 
determine the number of correct gestures the student has made. 

The letters and words recognized were also converted into speech for teachers to check if the gesture 
was right even without looking on the GUI. Additionally, the size of the characters was large enough to be 
seen by the students. 
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3.4. Testing of ASL alphabet and sign words 

Each of the letters in the ASL alphabetwas tested by doing the gestures once to check the accuracy 
of the device. The Table 1 shows that on the second and third trial, one letter was not recognized. Calculating 
for the rate of accuracy, E-Glove has 98% accuracy in detecting the letters. The simple formula below was 
used in determining the rate of accuracy of E-Glove in recognizing letters in the ASL alphabet: 


A= (Zii) « 100 (1) 


i=1 Yi 


The device was able to recognize all the letters on most of the trials but there were undetected letters 
due to conflicts in the reading because the mapped values of the flex sensors are closely related.When the 
conflict in reading happened, the E-Glove did not show any letters on the software program and the user had 
to repeat the gesture again. This implies that E-Glove is reliable because the software program will not show 
incorrect letters. 


Table 1. The Accuracy of E-Glove in recognizing letters in the ASL Alphabet 


Trial Number of Letters Number of Recognized Letters 
© Q x) 
1 26 26 
2 26 25 
3 26 26 
4 26 26 
5 26 25 


The same with the ASL alphabet, simple words were also tested. Accelerometer with gyroscope was 
used in the detection of words. Table 2 shows the number of recognized words for every trial. The E-Glove 
was not able to recognize all the words for the second and third trial. One wordwas missed.Using the simple 
formula for accuracy, testing the device achieved an accuracy of 98%. 

The device was only able to detect words with the right hand orientation or position fed in the 
software program. The recognition of words also exhibited the behavior of E-Glove when recognizing letters 
in which no words will be shown if gesture was undetected. This does not only to make E-Glove reliable in 
recognizing gestures for words but also let the users topractice the right orientation or positions of the hand 
when doing the gestures. 


Table 2. The Accuracy of E-Glove in recognizing ASL Sign Words 


Trial Number of Words Number of Recognized Words 
(Q) Q) Œ 
1 10 10 
2 10 9 
3 10 9 
4 10 10 
5 10 10 


The acceptable rate of accuracy achieved by the E-Glove suggests that the method of recognizing 
letters in which only one type of sensor was used compared to multiple sensors [1], [4] is sufficient to 
perform its function. Since fewer data will be read and processed, the performance of the device in terms of 
analyzing the gesture will be improved unlike the used of multiple sensors by previous studies where 
recognition speed was traded off to accuracy. Moreover, the number of flex sensors used was also reduced 
compared to other gloves which will make E-Glove cheaper to produce. 

In terms of recognizing ASL sign words, E-Glove was also able to show that with the use of a single 
accelerometer, an acceptable rate of accuracy can also be achieved. The accelerometer from previous 
studieswas used in combination with flex sensor [1], [4] to only recognize letters. 


4. CONCLUSION 
In deaf education using the BiBi approach, children learn American Sign Languageas a first 
language. They learn ASL as the primary way of communicating with other people. E-Glove works as an 
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automated translator and converts sign language directly into a vocal and textual format using flex sensors 
and accelerometer. Considering the accuracy it exhibited, the reduction of sensors to improve the gesture 
recognition speed and the cheaper productioncost, it can very well be implemented as a teaching aid for Deaf 
Education. Furthermore, the design of the GUI has an increased aesthetic appeal for children. This 
educational tool for hearing-impaired children would encourage a healthy classroom interaction. The E- 
Glove can further be improved by adding more ASL sign words from the ASL dictionary to its pool of 
words. The software program can also be developed in such a way that it can construct sentences from words 
detected. 
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